Streaming graph partitioning for large distributed graphs

Extracting knowledge by performing computations on graphs is becoming increasingly challenging as graphs grow in size. A standard approach distributes the graph over a cluster of nodes, but performing computations on a distributed graph is expensive if large amount of data have to be moved. Without partitioning the graph, communication quickly becomes a limiting factor in scaling the system up. Existing graph partitioning heuristics incur high computation and communication cost on large graphs, sometimes as high as the future computation itself. Observing that the graph has to be loaded into the cluster, we ask if the partitioning can be done at the same time with a lightweight streaming algorithm. We propose natural, simple heuristics and compare their performance to hashing and METIS, a fast, offline heuristic. We show on a large collection of graph datasets that our heuristics are a significant improvement, with the best obtaining an average gain of 76%. The heuristics are scalable in the size of the graphs and the number of partitions. Using our streaming partitioning methods, we are able to speed up PageRank computations on Spark [36], a distributed computation system, by 18 % to 39 % for large social networks

Location Services in Wireless Ad hoc and Hybrid Networks: A Survey

Location services are used in mobile ad hoc and hybrid networks either to locate the geographic position of a given node in the network or for locating a data item (content). One of the main usages of position location services is in location based routing algorithms. In particular, geographic routing protocols can route messages more efficiently to their destinations based on the destination node’s geographical position, which is provided by a location service. A content location service provides to the requesting node either the requested data itself or the identifier of the node that stores this data. Sometimes the position of the node that stores the data is also provided. Such data location services are useful for implementing content-sharing applications, cooperative caching, and publish subscribe systems. In this paper we present a taxonomy of location services and survey known techniques for constructing such a service in wireless ad hoc and hybrid networks

Probabilistic quorum systems in wireless ad hoc networks

Quorums are a basic construct in solving many fundamental distributed computing problems. One of the known ways of making quorums scalable and efficient is by weakening their intersection guarantee to being probabilistic. This paper explores several access strategies for implementing probabilistic quorums in ad hoc networks. In particular, we present the first detailed study of asymmetric probabilistic bi-quorum systems, that allow to mix different access strategies and different quorums sizes, while guaranteeing the desired intersection probability. We show the advantages of asymmetric probabilistic bi-quorum systems in ad hoc networks. Such an asymmetric construction is also useful for other types of networks with non uniform access costs (e.g, peer-to-peer networks). The paper includes both a formal analysis of these approaches backed up by an extensive simulation based study. In particular, we show that one of the strategies that uses Random Walks, exhibits the smallest communication overhead, thus being very attractive for ad hoc networks. Categories and Subject Descriptors: C.2.1 [Comp.-Communication Networks]: Network Architecture and Design—Wireless communication

A Lock-Free, Concurrent, and Incremental Stack Scanning for Garbage Collectors

Two major efficiency parameters for garbage collectors are the throughput overheads and the pause times that they introduce. Highly responsive systems need to use collectors with as short as possible pause times. Pause lengths have decreased significantly during the years, especially through the use of concurrent garbage collectors. For modern concurrent collectors, the longest pause is typically created by the need to atomically scan the runtime stack. All practical concurrent collectors that we are aware of must obtain a snapshot of the pointers on each thread’s runtime stack, in order to reclaim objects correctly. To further reduce the length of the collector pauses, incremental stack scans were proposed. However, previous such methods employ locks to stop the mutator from accessing a stack frame while it is being scanned. Thus, these methods introduce a potential long and unpredictable pauses for a mutator thread. In this work we propose the first concurrent, incremental, and lock-free stack scanning for garbage collectors, allowing high responsiveness and support for programs that employ fine-synchronization to avoid locks. Our solution can be employed by all concurrent collectors that we are aware of, it is lock-free, it imposes a negligible overhead on the program execution, and it supports the special in-stack references existing in languages like C#